Towards Efficient Algorithms for Approximating a Fuzzy Relation by Fuzzy Rules: Case When “And”- and “Or”-Operation are Distributive

A generic fuzzy relation often requires too many parameters to represent – especially when we have a relation between many different quantities x1, . . . , xn. There is, however, a class of relations which require much fewer parameters to describe namely, relations which come from fuzzy rules. It is therefore reasonable to approximate a given relation by fuzzy rules. In this paper, we explain how this can be done in an important case when “and”and “or”-operation are distributive – and we also explain why this case is important. I. FORMULATION OF THE PROBLEM Relations are ubiquitous.Many real-life quantities x1, . . . , xn are related – in the sense that once we know the value of one or more of the quantities, this knowledge restricts possible values of other quantities. In some cases, we have a functional relation – when the values of the quantities x1, . . . , xn−1 uniquely determine the value of the quantity xn. For example, according to Ohm’s law, the voltage V is uniquely determine by the current I and the resistance R as V = I ·R. In many other cases, however, we have relations which are not functional. In other words, even if we know the exact values of all the quantities x1, . . . , xn−1, we can still have different possible values of xn. This is actually true even for voltage: different materials exhibit minor deviation from the Ohm’s law; as a result, even if we know the current and the resistance, we can only conclude that the voltage V is close to I · R (e.g., that V can only takes values from the interval [I ·R− ε, I ·R+ ε] for some small ε > 0. In mathematical terms, a relation between real-valued quantities xi is usually defined as a mapping R : IR → {0, 1} such that R(x1, . . . , xn) = 1 indicates that the corresponding combination of values (x1, . . . , xn) is possible in a real-life situation. Real-life relations are often fuzzy. In practice, about some combinations (x1, . . . , xn), we are often not 100% sure whether these combinations are possible or not. In the traditional (“crisp”) approach, we simply count all such combinations as possible – since there is a possibility that such combinations occur. However, this crisp representation ignores the fact that we may be more certain about the possibility of some combinations and less certain about the possibility of others. To describe the different, it is necessary to know, for each possible combination (x1, . . . , xn), our degree of certainty that this combination is practically possible. In the computer, “true” is usually represented as 1, and “false” as 0. It is therefore natural to represent intermediate degrees of certainty as numbers from the interval [0, 1]: the larger the number, the larger our degree of confidence. The resulting mapping R : IR → {0, 1} is known as a fuzzy relation; see, e.g., [4], [10], [14]. Need for a concise representation of fuzzy relation. To use the information about the relation, we need to represent it in a computer. Theoretically, each of the quantities xi can have infinitely many different values, but in practice, due to inevitable measurement uncertainty, for each variable xi, we can only have finitely many distinguishable values xi1, . . . , xij , . . . , xiNi . Because of this, knowing a relation means that we know the values R(x1i1 , . . . , xnin) corresponding to all possible combinations (x1i1 , . . . , xnin). In principle, we can simply store the degrees of certainty corresponding to all possible N1 · . . . ·Nn combinations. This requires storing and processing ≈ N values, where N is a typical number of distinct values of each quantity. The problem with this representation is that, as we have mentioned, many quantities are related to each other; so, to have the most adequate representation of a real-life phenomenon, we need to describe a relation between a large number of variables. When n is large, the resulting number of values N grows exponentially with n – and, as it is well known about exponential functions, the numbers easily become astronomically high, exceeding the ability of modern computers to store and/or process this information; see, e.g., [11]. We therefore need to come up with a more concise representation of fuzzy relations. An approximate representation is OK. The degrees of certainty can only be approximately described: an expert cannot realistically distinguish between his/her degree of 0.71 and 0.72 :-) Since the values are only approximately known anyway, it is OK to represent them approximately. This possibility 978-1-4799-4562-7/14/$31.00 c ⃝2014 IEEE of using an approximate representation provides the flexibility which makes more concise representations possible. Fuzzy rules as a natural concise representation of fuzzy relations. Many fuzzy relations come from fuzzy rules, i.e., from a combination of rules of the type “if Ar,1(x1) and . . . and Ar,n−1(xn−1) then Ar,n(xn)”, where r = 1, . . . , nr is the number of the rule, and Ar,i(xi) are fuzzy properties. The ubiquity of such rules comes from the fact that this is how experts often describe their decisions. For example, a driver can explain his or her driving strategy by describing rules like “if a car in front is close, and it starts breaking seriously, one needs to hit the brakes hard right away”. Such rules use imprecise (fuzzy) words like “close”, “seriously”, “hard”, which are naturally described by fuzzy logic techniques. One of the most common ways to formalize the fuzzy rules it the Mamdani approach. In this approach, we take into account that a tuple (x1, . . . , xn) is consistent with the rules if for one of the given rules, the conditions are satisfied and the conclusion is satisfied as well. In other words, a tuple (x1, . . . , xn) is consistent with the given rules if and only if the following statement is true: (A1,1(x1)& . . . &A1,n−1(xn−1)&A1,n(xn)) ∨ . . .∨ (Anr,1(x1)& . . . &Anr,n−1(xn−1)&Anr,n(xn)). Fuzzy logic techniques enable us to transform this formula into the exact value of a degree d(x1, . . . , xn) to which the tuple (x1, . . . , xn) is consistent with the rules. Specifically: • we can use an “and”-operation (t-norm) f&(a, b) to represent “and”, and • we can use an “or”-operation (t-conorm) f∨(a, b) to represent “or”. As a result, we get the following degree: d(x1, . . . , xn) = f∨(d1(x1, . . . , xn), . . . , dnr (x1, . . . , xn)), where the degree dr(x1, . . . , xn) to which the tuple (x1, . . . , xn) is consistent with the r-th rule is equal to dr(x1, . . . , xn) = f&(Ar,1(x1), . . . , Ar,n−1(xn−1), Ar,n(xn)). Fuzzy rules are a natural concise way of representing a relation. Thus, it is reasonable to try to approximate a given fuzzy relation by an appropriate family of rules. What we do in this paper. In this paper, we propose new algorithms for representing a given fuzzy relation in terms of fuzzy rules, algorithms which are applicable in the important case when “and”and “or”-operations are distributive. II. “AND”AND “OR”-OPERATIONS (T-NORMS AND T-CONORMS) IN FUZZY LOGIC: BRIEF REMINDER Before we start describing our algorithms, we need to explain why the case when “and”and “or”-operations are distributive is important. To explain this importance, let us first recall the motivations behind the usual definitions of “and”operations (t-norms) and “or”-operations (t-conorms). Why t-norms and t-conorms: reminder. The main idea behind “and”-operations is that often, we know the expert’s degrees of confidence a = d(A) and b = d(B) in two statements A and B, and we want to estimate the expert’s degree of confidence in a composite statement A&B or A∨B. The only information that we have for this estimate consist of degrees a and b, so the resulting estimates are obtained by applying some computations to these two numbers: • the algorithm for producing the estimate for d(A&B) is denoted by f&(a, b), so the desired estimate has the form f&(d(A), d(B)); and • the algorithm for producing the estimate for d(A∨B) is denoted by f∨(a, b), so the desired estimate has the form f∨(d(A), d(B)). Which properties should the corresponding functions f&(a, b) and f∨(a, b) satisfy? Commutativity. The composite statements A&B and B&A are equivalent to each other for every two statements A and B. It is therefore reasonable to require that the estimates f&(a, b) and f&(b, a) for the expert’s degree of belief in these composite statements coincide, i.e., that f&(a, b) = f&(b, a). In mathematical terms, this means that the “and”-operation f&(a, b) should be commutative. Similarly, since for every two statements A and B, the composite statements A∨B and B ∨A are equivalent to each other, it is reasonable to require that the estimates f∨(a, b) and f∨(b, a) for the expert’s degree of belief in these composite statements coincide, i.e., that f∨(a, b) = f∨(b, a). Thus, the “or”-operation f∨(a, b) should also be commutative. Associativity. Another pairs of equivalent statements are (A&B)&C and A&(B&C). We can estimate the expert’s degree of belief in the statement (A&B)&C if: • first, we apply the “and”-operation to the degrees a = d(A) and b = d(B) and get an estimate f&(a, b) for the expert’s degree of belief in a statement A&B; • then, we apply the same “and”-operation to another pair of numbers: ◦ our estimate f&(a, b) of the expert’s degree of belief in A&B, and ◦ the expert’s degree of belief c = d(C) in the statement C. As a result, we get the estimate f&(f&(a, b), c) for the expert’s degree of belief in (A&B)&C. Similarly, we can estimate the expert’s degree of belief in the statement A&(B&C) if: • first, we apply the “and”-operation to the degrees b = d(B) and c = d(C) and get an estimate f&(b, c) for the expert’s degree of belief in a statement B&C; • then, we apply the same “and”-operation to another pair of numbers: ◦ the expert’s degree of belief a = d(A) in the statement A; and ◦ our estimate f&(b, c) of the expert’s degree of belief in B&C. As a result, we get the estimate f&(a, f&(b, c)) for the expert’s degree of belief in A&(B)&C). Since the statements (A&B)&C and A&(B&C) are equivalent A&(B∨C) and (A&B)∨(A&C), it is reasonable to require that the corresponding estimates f&(f&(a, b), c) and f&(a, f&(b, c)) for the expert’s degrees of belief in these statements be equal, i.e., that f&(f&(a, b), c) = f&(a, f&(b, c)) for all a, b, and c. In mathematical terms, this means that the “and”-operation f&(a, b) should be associative. Similarly, since the composite statements (A∨B)∨C and A ∨ (B ∨ C) are equivalent to each other, it makes sense to require that the corresponding estimates f∨(f∨(a, b), c) and f∨(a, f∨(b, c)) for the expert’s degrees of belief in these statements be equal, i.e., the “or”-operation f∨(a, b) should also be associative. Because of associativity, we can simply write f&(a, b, . . . , c) and f∨(a, b, . . . , c) without worrying about the order of the corresponding “and”and “or”-operations. Other properties of “and”and “or”-operations. Other properties of “and”and “or”-operations also follow from common sense. For example, from the fact that “true”&A is equivalent to A, we conclude that f&(a, 1) = a. From the fact that “true”∨A is equivalent to “true”, we conclude that f∨(a, 1) = 1. Similarly, from the fact that “false”&A is equivalent to “false”, we conclude that f&(a, 0) = a, and from the fact that “false”∨A is equivalent to A, we conclude that f∨(a, 0) = a. Another example: if an expert increases his/her belief in one or both of the statements A and B, then it is reasonable to assume that the expert’s degree of belief in a composite statement A&B will either increase or stay the same, but it cannot decrease. In other words, if a ≤ a′ and b ≤ b′, then we should have f&(a, b) ≤ f&(a, b′). In mathematical terms, this means that the “and”-operation should be a (non-strictly) increasing function of each of its variables. Similarly, it is reasonable to require that the “or”-operation f∨(a, b) is a non-strictly increasing function of each of its variables. It is also reasonable to require that small changes in degree a = d(A) and b = d(B) should lead to small changes in d(A&B). In other words, it is reasonable to require that the “or”-operation be continuous. III. WHAT ABOUT DISTRIBUTIVITY? Distributivity: reminder. In the previous section, when we described “and”and “or”-operations, we considered only equivalences which use only one of the two connectives: either “and” or “or”. In logic, there are also equivalences which combine both “and” and “or”. One of these properties is distributivity. Specifically, for every three statements A, B, and C, the composite statements A&(B ∨ C) and (A&B) ∨ (A&C) are equivalent to each other. Seemingly natural idea: let us add distributivity to the list of requirements on “and”and “or”-operations. In the previous sections, we showed that: • from the fact that A&B is equivalent to B&A, we implied that the “and”-operation should be commutative; • from the fact that A&(B&C) is equivalent to (A&B)C, we implied that the “and”-operation is associative. Similar equivalences about ∨ led us to commutativity and associativity of “or”-operations. It seems reasonable to see what we can conclude based on the fact that the statements A&(B∨C) and (A&B)∨(A&C) are equivalent to each other. Similar to the previous section, we will see what estimates we get when we use “and”and “or”-operations to estimate the degree of confidence in both expressions, and we will require that the resulting estimates coincide. We can estimate the expert’s degree of belief in the first statement A&(B ∨ C) if: • first, we apply the “or”-operation to the degrees b = d(B) and c = d(C) and get an estimate f∨(b, c) for the expert’s degree of belief in a statement B ∨ C; • then, we apply the “and”-operation to the following pair of numbers: ◦ the expert’s degree of belief a = d(A) in the statement A, and ◦ our estimate f∨(b, c) of the expert’s degree of belief in B ∨ C. As a result, we get the estimate f&(a, f∨(b, c)) for the expert’s degree of belief in A&(B ∨ C). Similarly, we can estimate the expert’s degree of belief in the second statement (A&B) ∨ (A&C) if: • first, we apply the “and”-operation to the degrees a = d(A) and b = d(B) and get an estimate f&(a, b) for the expert’s degree of belief in a statement A&B; • then, we apply the same “and”-operation to the degrees a = d(A) and c = d(C) and get an estimate f&(b, c) for the expert’s degree of belief in a statement B&C; • finally, we apply the “or”-operation to the following pair of numbers: ◦ our estimate f&(a, b) of the expert’s degree of belief in A&B, and ◦ our estimate f&(a, c) of the expert’s degree of belief in A&C. As a result, we get the estimate f∨(f&(a, b), f&(a, c)) for the expert’s degree of belief in (A&B) ∨ (A&C). Since the statements A&(B ∨ C) and (A&B) ∨ (A&C) are equivalent to each other, it is reasonable to require that the corresponding estimates f&(a, f∨(b, c)) and f∨(f&(a, b), f&(a, c)) for the expert’s degrees of belief in these statements be equal, i.e., that f&(a, f∨(b, c)) = f∨(f&(a, b), f&(a, c)) for all a, b, and c. In mathematical terms, this means that the “and”-operation f&(a, b) should be distributive over the “and”-operation f∨(a, b). Good news: there are reasonable distributive pairs of “and”and “or”-operations. Let us show that some reasonable pairs of “and”and “or”-operations do have the distributivity property. The example is when we use f∨(a, b) = max(a, b) (one of the most frequently used “or”-operations) and an arbitrary “and”-operation f&(a, b). Let us show that in this case, we have distributivity, i.e., that f&(a,max(b, c)) = max(f&(a, b), f&(a, c)) for all a, b, and c. Indeed, without losing generality, we can assume that b ≤ c (when c ≤ b, distributivity can be proven in the exact same way). In this case, max(b, c) = c, so the left-hand side of the desired equality is equal to f&(a, c): f&(a,max(b, c)) = f&(a, c). Due to the fact that the “and”-operation is increasing, b ≤ c implies that f&(a, b) ≤ f&(a, c). Thus, the right-hand side of the desired equality is also equal to f&(a, c): max(f&(a, b), f&(a, c)) = f&(a, c). So, both sides of the desired equality are equal to the same value and are, thus, equal to each other. Not so good news: distributivity requirement excludes many reasonable “or”-operations. The above example looks great, but it turns out that this is the only such example. Indeed, for an “and”-operation, we have f&(1, a) = a and f∨(a, 1) = 1 for all a. In particular, for b = c = 1, we have f∨(b, c) = 1, f&(a, b) = a, and f&(a, c) = a. Thus, the left-hand side of the distributivity equality is equal to f&(a, f∨(b, c)) = f&(a, 1) = a, while the right-hand side is equal to f∨(f&(a, b), f&(a, c)) = f∨(a, a). Thus, for b = c = 1, distributivity implies that f∨(a, a) = a for every a. One can show that that the only “or”-operations satisfying this condition is f∨(a, b) = max(a, b). Indeed, if b ≤ a, then from the known property f∨(a, 0) = a, new property f∨(a, a) = a, and monotonicity a = f∨(a, 0) ≤ f∨(a, b) ≤ f∨(a, a) = a, we conclude that f∨(a, b) = a. Similarly, for b ≥ a, we get f∨(a, b) = b. In both cases, we get f∨(a, b) = max(a, b). Why this is a problem. While the maximum “or”-operations works well in many cases, in many other situations, other “or”operations work better. For example, in fuzzy control [6], [12]: • if we are interested in the smoothest control, then we should select f∨(a, b) = max(a, b); • however, if we are interested in the most stable control, then we should select f∨(a, b) = a+ b− a · b. Similar, if we look for an operation f∨(a, b) which is the least sensitive to the uncertainty with which we can estimate the original degrees of belief a and b, then [6], [8], [10]: • if we minimize the worst-case uncertainty in f∨(a, b), we get f∨(a, b) = max(a, b); • however, if we instead minimize the average uncertainty, we get f∨(a, b) = a+ b− a · b. If we select an “and”-operation based on the principle of maximum entropy [3] – a natural formalization of Laplace’s principle of indifference – we also get f∨(a, b) = a+ b− a · b [6], [7]. So what do we do: need to consider limited distributivity. We would like to require distributivity and at the same time still allow “or”-operations which are different from maximum. Since full distributivity does not allow such “or”-operations, a natural idea is to consider limited distributivity. The above proof that only f∨(a, b) = max(a, b) leads to distributivity is based on considering pairs b and c for which f∨(b, c) = 1. It is therefore reasonable to only consider cases when f∨(b, c) < 1. Thus, we arrive at the following definition. Definition 1. We say that a pair of an “and”-operation f&(a, b) and an “or”-operation f∨(a, b) are distributive if for every three real numbers a, b, and c, f∨(b, c) < 1 implies f&(a,max(b, c)) = max(f&(a, b), f&(a, c)). In the following text, this is how we will understand distributivity of “and”and “or”-operations. Comment. It is worth mentioning that when f∨(b, c) < 1, then both sides of the distributivity equality, i.e., both values f&(a,max(b, c)) and max(f&(a, b), f&(a, c)), are smaller than 1. Indeed, due to monotonicity, we have f&(a, f∨(b, c)) ≤ f&(1, f∨(b, c)) = f∨(b, c) < 1, so the left-hand side is indeed smaller than 1. Due to monotonicity, f&(a, b) ≤ f&(1, b) = b and f&(a, c) ≤ f&(1, c) = c. Thus, due to monotonicity, we have f∨(f&(a, b), f&(a, c)) ≤ f∨(b, c) < 1. So, the right-hand side is also smaller than 1. Is the above limitation sufficient? A new example of operations which are distributive in the above sense. Is the above restriction sufficient? Do we have now “or”-operations beyond maximum? The following example shows that the answer to both questions is “yes”. To explain this example, let us recall that the notion of distributivity started with arithmetic, where multiplication is distributive with respect to addition: a · (b+ c) = a · b+ a · c. It is therefore reasonable to consider an example, in which the “and”-operation is multiplication and the “or”-operation is addition. Multiplication f&(a, b) = a · b (“algebraic product”) is indeed one of the most frequently used “and”-operations. In contrast, pure addition a+b cannot be an “or”-operation, since: • an “or”-operation, given two values a, b ∈ [0, 1], should always return a value f∨(a, b) ∈ [0, 1], • while for numbers a, b ≤ 1, the sum a+b can be larger than 1: e.g., when a = b = 1, we have a+ b = 2 > 1. Once we restrict the sum to 1 from above, i.e., consider the operation f∨(a, b) = min(a + b, 1), then we already get one of the most frequently used “or”-operations. In this case, if we limit ourselves to situations when the “or”-operation coincides with addition, i.e., when b + c < 1, then f∨(b, c) = b + c, so the left-hand side of the desired equality takes the form f&(a, f∨(b, c)) = a · f∨(b, c) = a · (b+ c). For the right-hand side, we get f&(a, b) = a ·b and f&(a, c) = a · c. From b+ c ≤ 1 and a ≤ 1, it follows that a · (b+ c) = a · b+ a · c ≤ 1. Thus, we have f∨(f&(a, b), f&(a, c)) = f∨(a · b, a · c) = a · b+ a · c. The equality between the expressions for the left-hand side and the right-hand sides now follows from the well-known fact that multiplication is distributive with respect to addition. Resulting proposal: let us restrict ourselves to distributive “and’and “or”-operations. Instead of considering arbitrary pairs of “andand “or”-operations, we propose to consider only pairs which are distributive (in the sense of Definition 1). The motivation for this proposal is the same as the motivation for “or”and “and”-operations to be commutative and distributive. Comment. Of course, as we have mentioned, even when we know the degrees of belief a and b in statements A and B, the value f&(a, b) only approximately describes our degree of belief in A&B. Because of the approximate character of “and”and “or”-operations, it is reasonable to expect that the empirical “and”and “or”-operations are only approximately associative – and this has indeed been discovered when researchers analyzed how people think; see, e.g., [5], [13], [15]. Similarly, we expect that the empirical “and”and “or”operations are only approximately distributive. Another reason why associativity – and, similarly, distributivity – can be only approximate is that in fuzzy control, while we start with “and”and “or”-operations which best describe expert’s reasoning, we eventually need to switch to operations which provide the best control (smoothest, most stable, etc.). It is known that this need to go beyond the original expert ideas, to an event better control, sometimes leads to more complex operations which may not necessary be exactly associative (or distributive); see, e.g., [12]. IV. HOW TO DESCRIBE DISTRIBUTIVE “AND”AND “OR”-OPERATIONS Need for a general description. The main objective of this paper is to approximate a general fuzzy relation by fuzzy rules. In the previous section, we explained why it is reasonable to require that the “and”and “or”-operations are distributive. In view of this explanation, our goal is to approximate a general fuzzy relation by fuzzy rules that use distributive “and”and “or”-operations. We would like to produce an algorithm which is applicable for each distributive pairs of operations. So, to approach this approximate problem, let us see how we can describe such a generic pair. To come up with such a description, let us first recall how we can describe a generic “or”-operation. Different types of “or”-operations: reminder. Some of the “or”-operations are Archimedean, in the sense that for every two values a ∈ (0, 1) and b ∈ (0, 1) for which f∨(a, a, . . . , a) (n times) > b. A typical example of an Archimedean “or”-operation is the “algebraic sum” f∨(a, b) = a+ b−a · b. From the algebraic viewpoint, it is known that all such operations are isomorphic to addition, i.e., there exists a function ψ : [0, 1] → [0,∞] such that f∨(a, c) = c if and only if ψ(a)+ψ(b) = ψ(c); see, e.g., [4], [10]. In other words, each such operation has the form f∨(a, b) = ψ−1(ψ(a) + ψ(b)), where ψ−1(t) denotes the function which is inverse to the function ψ(a). Another type of “and”-operations is an operation which is isomorphic to f∨(a, b) = min(a+b, 1), i.e., an operation of the type f∨(a, b) = ψ−1(min(ψ(a) + ψ(b), 1)), for some strictly increasing function ψ(a). We also have f∨(a, b) = max(a, b). It is known (see, e.g., [4], [10]) that every “or”-operation is isomorphic to a lexicographic combination of Archimedean operations, isomorphic to f∨(a, b) = min(a+ b, 1), and max. The exact description of a generic “or”-operation is complex, but we can consider approximate descriptions. As we have mentioned earlier, the main purpose of “or”-operation f∨(a, b) is to estimate the expert’s degree of belief d(A∨B) in a composite statement A∨B as f∨(d(A), d(B)). It is therefore reasonable to select an “or”-operation for which, for all the pairs of statements (Ak, Bk) for which we know both the degrees d(Ak) and d(Bk) and the actual expert’s degree of belief d(Ak∨Bk), we should have d(Ak∨Bk) ≈ f∨(d(Ak), d(Bk)). Because of the approximate character of an “or”-operation, we can always replace it with a very close one without changing the practical accuracy of the approximation. For example, if an estimate f∨(d(Ak), d(Bk)) approximates the actual expert’s degree d(Ak ∨ Bk) with an accuracy of 10%, then replace the corresponding “or”-operation with another operation f ′ ∨(a, b) ≈ f∨(a, b) one which is 0.01-close (or even 0.001-close) to f∨(a, b), we get, in effect, the exact same approximation accuracy. To be more precise, for every ε > 0, we say that an “or” operation f ′ ∨(a, b) is an ε-approximation to an “or”-operation f∨(a, b) if for all a and b, we have |f ′ &(a, b)− f&(a, b)| ≤ ε. Approximate descriptions can indeed be simpler. It is known (see, e.g., [9]) that for every “or”-operation f∨(a, b) and for every ε > 0, there exists an Archimedean “or”-operation f ′ ∨(a, b) which is ε-close to f∨(a, b). Alas, the resulting approximation does not help in describing generic distributive pairs. Maybe we can use this universal approximation result to describe distributive pairs of “and”and “or”operations? Alas, no. In this approximation result, we approximate each “or”operation by an Archimedean one. For each Archimedean operation, if b < 1 and c < 1, then we have f∨(b, c) < 1. Thus, due to our definition of distributivity, we would have distributivity for all b < 1 and c < 1 and thus, by continuity, for all b and c, and we already know that this is only possible for f∨(a, b) = max(a, b) – which is not an Archimedean “or”operation. Thus, if we want to require distributivity, we need to consider non-Archimedean “or”-operations. A new universal approximation result. The above universal approximation result says that each “or”-operation f∨(a, b) can be approximated by an Archimedean “or”-operation f ′ ∨(a, b). We have already mentioned that each Archimedean “or”operation has the form f ′ ∨(a, b) = ψ −1(ψ(a) + ψ(b)) for some function ψ : [0, 1] → [0,∞]. We want to find a nonArchimedean approximation to f ′ ∨(a, b), which will then be an approximation to the original t-norm f∨(a, b). Indeed, for every δ > 0, we can consider a new function ψ′(a) which is equal to ψ(a) for all a ≤ 1 − δ and which is equal to ψ(1 − δ) + (a − (1 − δ)) for all a ∈ (1 − δ, 1]. For this function ψ′(a), the operation f ′′ ∨(a, b) def = (ψ′)−1(min(ψ′(a) + ψ′(b), ψ′(1)) is an “or”-operation, and one can prove that when δ → 0, we have f ′′ ∨(a, b) → f ′ ∨(a, b). Thus, for sufficiently small δ > 0, the new operation f ′′ ∨(a, b) is indeed an approximation to f ′ ∨(a, b) and thus, to the original “or”-operation f∨(a, b). We can rewrite the above expression for f ′′ ∨(a, b) in a more familiar form if we take ψ′′(a) def = ψ′(a) ψ′(1) . One can show that in terms of this new function ψ′′(a), the “and”-operation has the form f ′′ ∨(a, b) = (ψ ′′)−1(min(ψ′′(a) + ψ′′(b), 1)). We therefore conclude that for every ε > 0, each “or”operation can be ε-approximated by by an “or”-operation which is isomorphic to min(a+ b, 1). Let us use the new approximation result to get a general description of distributive pairs. Under the above approximation result, let us now describe all distributive pairs of fuzzy logic operations. In this description, we assume that the “or”-operation is isomorphic to f∨(a, b) = min(a + b, 1). This means that if we “re-scale” all the original degrees of belief a, b, c ∈ [0, 1] into values a′ = ψ′′(a), b′ = ψ′′(b), and c′ = ψ′′(c), then the original relation c = f∨(a, b) takes a simplified form c′ = min(a′ + b′, 1). We can apply the same re-scaling to the “and”-operation f&(a, b), resulting in a new “and”-operation g(a′, b′) def = ψ(f&((ψ ′′)−1(a′), (ψ′′)−1(b′)). One can easily check that this is indeed an “and”-operation (i.e., a t-norm). In the new scale, our distributivity condition takes the following form: if b′ + c′ < 1, then g(a′, b′ + c′) = g(a′, b′) + g(a′, c′). In other words, for each a′, the function b′ → g(a′, b′) is a monotonic additive function of b′. It is known [1] that all monotonic additive functions have the form f(x) = k · x. Thus, we have g(a′, b′) = k(a′) · b′ for some k(a′). Since every “and”-operation is commutative g(a′, b′) = g(b′, a′), we get k(a′) · b′ = k(b′) · a′. Dividing both sides of this equality by a′ · b′, we conclude that k(a′) a′ = k(b′) b′ . In other words, we conclude that the ratio k(a′) a′ has the same value for all possible values a′ ∈ [0, 1] – in other words, we conclude that this ratio is a constant. Let us denote this constant by r. Then, from k(a′) a′ = r, we conclude that k(a′) = r · a′. Therefore, g(a′, b′) = k(a′) · b′ = r · a′ · b′. From the requirement that g(1, 1) = 1, we conclude that r = 1 and thus, g(a′, b′) = a′ · b′. So, we arrive at the following conclusion. Resulting description of a generic pair of distributive operations. Each “or”-operation can be, with arbitrary accuracy, approximated by an operation isomorphic to min(a + b, 1). Thus, for all practical purposes, we can assume that the actual “or”-operation is isomorphic to min(a+ b, 1). Under this assumption, each distributive pair is isomorphic to the pair consisting of an “and”-operation f∨(a, b) = min(a + b, 1) and the algebraic-product ”and”-operation f&(a, b) = a · b. V. APPROXIMATING A FUZZY RELATION BY FUZZY RULES: WHAT WE PROPOSE The problem of approximating a fuzzy relation: reminder. Now that we know how to describe a general distributive pair of “and”and “or”-operations, we can handle the original approximation problem: we have a fuzzy relation R(x1, . . . , xn), we have a distributive pair of “and”and “or”-operations, and we want to represent it as R(x1, . . . , xn) = f∨(d1(x1, . . . , xn), . . . , dnr (x1, . . . , xn)),